An analysis of standardized reading ability tests: What do questions actually measure?

This study examined what types of passage and individual item attributes of reading ability tests affect item difficulty. Ninety-six questions from the GatesMacGinitie Reading Test for 7/9th graders were analyzed in terms of individual item characteristics and passage features that were considered to affect the item’s difficulty. Whereas two passage features, word frequency and sentence length, influenced the item difficulty, none of the individual question characteristics were found to affect the item difficulty. Thus, the GMRT may be assessing how difficult a passage they can comprehend, not how difficult a question they can answer about a passage. These findings may be the result of the passages in the GMRT having an atypical construction; shorter passages tended to be composed of longer sentences.